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FOREWORD 

This  research  and  development  effort  was  conducted  in  response  to  Navy  Decision 
Coordinating  Paper,  Personnel  Supply  Systems  (NDCP-ZO 1 07-PN)  under  subproject  PN.I2, 
Officer  Management  Systems,  and  the  sponsorship  of  the  Deputy  Chief  of  Naval 
Operations  (OP-OI).  The  objective  of  the  subproject  is  to  develop  a set  of  user-oriented, 
computer-based  models  and  techniques  to  assist  in  the  development  of  a Navy  officer 
force  that  meets  its  manpower  requirements.  This  report  describes  improved  techniques 
for  forecasting  loss  rates  for  the  Navy's  unrestricted  line  (URL)  officer  community.  The 
results  of  this  work  are  now  being  used  by  OP- 1 30,  Officer  Programs  Implementation 
Branch,  on  a regular  basis. 

Acknowledgments  are  due  to  LCDR  D.  Parker  and  LT  D.  Burns,  OP- 1 30,  for 
assistance  in  data  collection  and  analysis. 


DONALD  F.  PARKER 
Commanding  Officer 
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SUMMARY 


Problem 


The  Navy  uses  estimates  of  unrestricted  line  (URL)  officer  losses  (by  type  and  grade) 
to  plan  promotions  and  accessions.  Without  accurate  estimates,  these  plans  could  yield  an 
officer  force  that  does  not  meet  authorized  strengths. 

Objective 

The  purpose  of  this  effort  was  to  compare  the  accuracy  of  an  existing  URL  loss  rate 
forecasting  method  with  that  of  other  techniques  and  to  recommend  an  improved 
technique  for  estimating  losses  in  each  grade/length  of  service  category. 

Approach 

A number  of  estimating  techniques  were  tested  to  forecast  loss  rates  for  the  total 
URL.  These  techniques  used  historical  loss  rate  data  (FY  1969-1977)  by  grade,  years  of 
commissioned  service  (YCS),  and  promotion  status.  Estimates  generated  for  each 
technique  were  compared  and  the  "best"  techniques  chosen  on  the  basis  of  their  relative 
forecasting  accuracy  over  time. 

Findings 


1.  Autoregressive  time  series  methods  produced  generally  more  accurate  estimates 
than  simple  or  weighted  moving  averages  when  judged  with  a minimum  mean  absolute 
error  (MAE)  criterion. 

2.  Minimum  absolute  deviation  (MAD)  regression  produced  better  autoregressive 
time  series  estimates  than  ordinary  least  squares  (OLS)  regression. 

3.  The  best  autoregressive  technique,  a 3-year  MAD  approach,  reduced  the  forecast 
error  of  the  previous  technique  over  all  grade/length  of  service  combinations  by  65 
percent. 


4.  The  autoregressive,  3-year  MAD  forecasting  technique  produces  substantially 
more  accurate  loss  rate  estimates  than  the  existing  OP- 1 30  method. 

Conclusions 


Although  other  techniques  may  produce  better  estimates  in  isolated  cases,  the 
autoregressive,  3-year  MAD  technique  can  conveniently  be  used  in  all  cases  with  little 
loss  of  accuracy. 
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INTRODUCTION 


Problem 


Within  the  Navy's  officer  personnel  system,  the  expected  number  of  vacancies  at  a 
particular  grade  (rank)  determines  the  number  of  officers  that  will  be  (I)  promoted  to  that 
grade  and  all  lower  grades,  (2)  gained  from  a variety  of  commissioning  sources,  and  (3) 
transferred  from  a reserve  to  a regular  commissioned  status  (a  process  called 
augmentation).  Vacancies  occur  when  the  number  of  officers  in  a particular  grade  falls 
below  the  authorized  strength  for  that  grade.  Vacancies  are  created  when  officers  are 
promoted  to  the  next  grade  or  when  officers  leave  the  Navy  (e.g.,  retire,  resign,  die,  or 
leave  through  a variety  of  administrative  actions).  Without  an  accurate  estimate  of  losses 
within  each  grade,  plans  for  subsequent  promotion,  accession,  and  augmentation  actions 
are  likely  to  fall  short  of  or  exceed  authorized  strength. 

Background 

The  Officer  Program  Implementation  Branch  of  the  Deputy  Chief  of  Naval 
Operations  (Manpower)  (OP- 1 30)  operates  the  Projected  Officer  Personnel  Inventory 
(POPI)  model,  in  part,  to  project  unrestricted  line  officer  (URL)  personnel  inventories 
based  on  predicted  loss  behavior.  These  projections  are  used  to  help  derived  promotion, 
augmentation,  and  accession  plans. 

Losses  are  forecast  for  each  URL  grade,  years  of  commissioned  service  (YCS),1  and 
promotion  status  (PS)2  "cell"  by  applying,  for  each  cell,  an  estimated  loss  rate  to  the 
inventory  at  the  beginning  of  the  planning  period.  The  rate  indicates  the  proportion  of 
officers  from  an  inventory  that  can  be  expected  to  be  lost  during  the  period. 

Loss  rate  forecasts  are  needed  not  only  for  the  URL,  but  also  for  individual  specialty 
areas  within  the  URL  (e.g.,  surface,  submarine)  and  for  the  restricted  line  (RL)  and  Staff 
Corps.  Forecasted  rates  are  used  in  POPI,  an  accession  planning  model,  a steady-state 
inventory  projection  model,  and  in  studying  the  inventory  structure  of  various  speciality 
groups  under  a variety  of  scenarios. 

The  loss  rate  forecasts  formerly  used  in  these  models  and  studies  were  generated 
from  weighted  averages  of  historical  loss  rates.  After  several  years  of  using  this  method, 
OP- 1 30  felt  that  an  improved  method  was  needed  for  estimating  losses  for  all  grade,  YCS, 
and  PS  cells. 


1 Years  of  commissioned  service  are  computed  from  an  officer's  relative  year  group. 
The  year  group  is  the  fiscal  year  of  first  commissioning.  An  officer's  relative  year  group 
is  then  the  number  of  years  between  the  current  fiscal  year  and  that  first  commissioning. 
Hence,  relative  year  group  serves  as  a measure  of  length  of  (commissioned)  service. 

2Promotion  status  refers  to  whether  or  not  an  officer  has  failed  to  be  selected  for 
promotion  to  the  next  higher  grade.  Loss  behavior  tends  to  be  significantly  different  for  a 
LCDR  or  CDR  who  has  failed  selection  to  CDR  or  CAPT,  respectively,  than  for  one  who 
has  not  failed  ("due  course").  As  a result,  historical  loss  trends  and  forecasts  for  "due 
course"  and  "failed  selection"  officers  must  be  studied  separately. 


Objective 


The  purpose  of  this  effort  was  to  compare  the  accuracy  of  the  existing  URL  loss  rate 
forecasting  method  with  that  of  other  techniques  and  to  recommend  a "best"  technique 
for  each  grade/ YCS  cell.3 


3Henceforth,  grade  will  refer  to  rank  as  well  as  to  promotion  status. 
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APPROACH 


A number  of  techniques  were  tested  in  an  attempt  to  forecast  reasonably  accurate 
loss  rates  for  the  total  URL.  Loss  rates  were  forecasted  and  evaluated  by  grade  (Ensign 
through  Captain),  years  of  commissioned  service  (YCS)  (measured  by  relative  year  group), 
and  promotion  status.  Table  I displays  the  eight  grades  for  which  estimates  were  made 
and  their  relevant  YCS.  Each  of  the  techniques  employed  assume  that  projected  loss 
rates  for  a cell  depend  only  on  past  rates  for  that  cell.  No  attempt  was  made  to  relate 
loss  rates  to  exogenous  variables  (e.g.,  private  sector  employment  behavior)  or  individual 
officer  attributes  (e.g.,  education  levels). 


Table  I 

Grades/Years  of  Commissioned  Service  (YCS)  for 
Which  Loss  Rate  Estimates  Were  Produced 


Grade 

YCS 

Ensign  (ENS) 

0-1 

Lieutenant  JG  (LTJG) 

1-5 

Lieutenant  (LT) 

3-1  1 

Lieutenant  Commander  (LCDR) 

Lieutenant  Commander-Failed  for  Selection 

8-15 

to  Commander  (LCDRF) 

14-20 

Commander  (CDR) 

14-21 

Commander-Failed  for  Selection  to  Captain  (CDRF) 

20-26 

Captain  (CAPT) 

20-29 

Note.  These  YCS  "zones"  tend  to  represent  length  of  service  typical  for  "due  course" 
and  "fail  select"  officers.  "Early  select"  officers  are  not  broken  out  in  this  analysis. 


After  loss  rate  estimates  were  generated  by  each  technique,  they  were  compared  and 
the  "best"  techniques  chosen  on  the  basis  of  their  forecasting  accuracy  over  time. 

Nine  years  of  historical  loss  rate  data  (FY69  through  FY77,  excluding  FY7T)  were 
used  in  the  computations.  These  data  were  taken  from  the  Attrition  Data  Base  (ADB) 
maintained  by  OP-OI.  In  the  ADB,  loss  rates  are  generated  each  fiscal  year  by  dividing 
the  total  number  of  losses  in  a grade/YCS  cell  by  the  beginning  inventory  in  the  cell. 
Losses  in  the  ADB  are  defined  in  Table  2. 

Techniques  Examined 

A plot  of  the  nine  annual  observations  for  each  grade/YCS  cell  indicated  initially  that 
several  techniques  or  models  might  explain  loss  rate  behavior.  Some  of  the  series 
displayed  seemingly  constant  trends,  others  appeared  to  have  increasing  or  decreasing 
linear  trends,  while  still  others  suggested  exponentially  increasing  or  decreasing  behavior 
over  at  least  part  of  the  observed  period.  This  examination  led  to  the  test  of  the 
techniques  described  below: 
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Table  2 


Losses  by  Type  and  BUPERS  Loss  Code 


Type  of  Loss 

BUPERS  Loss  Code 

Retirement 

891-898 

Released  from  Active  Duty  (RAD) 

890 

Resignation 

701-705,  810-81 1,  889,  912 

Death 

870 

Discharge 

801-807,  899,  903,  905 

Community  Change 

997 

aSignifies  change  from  one  "community"  to  another  (e.g.,  from  URL  to  RL,  from 
aviation  to  surface  warfare)  depending  on  how  "communities"  are  defined. 


I.  "Naive"  or  Previous  Year  (PY):  The  simplest  of  all  forecasting  techniques  is  to 
suggest  that  next  year's  loss  rate  will  be  the  same  as  this  year's  rate.  This  naive  approach 
is  described  by 


t+1 


= X 


(1) 


where  X.  is  the  loss  rate  for  year  t,  and  Xf+|  is  the  estimated  loss  rate  for  year  t + I.  It 
should  be  noted,  however,  that  the  naive  model  is  reliable  only  if  the  data  displays  a 
rather  constant  trend. 

2.  Simple  (Unweighted)  Moving  Averages  (SM):  A second  approach  was  to  take  a 
simple  or  unweighted  moving  average  of  some  specified  number  of  past  year's  loss  rates  as 
a forecast  for  the  current  year.  The  forecast  is  then 

n 

I xt_i+l  (2) 

X = * 

Xt+1  n 


In  order  to  smooth  the  data  (i.e.,  to  remove  any  extreme  outliers),  this  method  was 
modified  as  follows:  The  rates  used  for  each  estimate  were  compared  to  the  mean  of 
those  rates — any  data  point  falling  outside  of  plus  or  minus  (±)  one  standard  deviation 
from  that  mean  was  removed  from  the  moving  average. 

Five  variations  of  this  method  were  tried,  using  from  3 to  7 years  of  data  to  estimate 
the  following  year's  rate.  Although  the  SM  approach  gives  relatively  accurate  estimates 
(i.e.,  the  estimates  minimize  the  sum  of  squared  error  from  historical  data),  these 


k 


estimates  are  not  responsive  to  recent  data.  As  the  number  of  observations  used  (n) 
increases,  the  stability  of  the  estimate  increases,  but  the  estimate  will  take  n years  to 
respond  fully  to  an  abrupt  change  in  data.  Since  a dynamic  process  is  being  modelled,  it 
may  be  more  appropriate  to  use  larger  weights  for  recent  data  than  those  used  for  older 
data.1*  This  consideration  leads  to  the  next  method. 

3.  Weighted  Moving  Averages  (WM):  Intuitively,  loss  behavior  for  a grade/YCS  cell 
should  resemble  the  recent  past  more  than  the  distant  past,  but  the  older  observations 
may  still  have  some  influence.  A weighted  moving  average  with  larger  weights  on  recent 
data  will  reflect  this  emphasis.  The  prior  OP- 1 30  method  of  loss  rate  forecasting  used  a 
weighted  moving  average  of  this  typer 


n X ...  . (n-i+1) 

x. - z 1 


t+1 


i=l 


n 

l i 

i=l 


(3) 


This  method  was  also  modified  to  exclude  "outliers"  in  a fashion  similar  to  the  SM  method. 

As  with  the  previous  method,  five  versions  of  the  SM  method  were  tried,  using  from  3 
to  7 years  of  data  to  estimate  the  next  year's  data. 

Both  the  SM  and  WM  methods  are  computationally  convenient  but  they  lack  a 
methodology  for  selecting  an  appropriate  weighting  structure.  For  this  reason,  two 
additional  types  of  weighting  models  that  attempt  to  choose  the  appropriate  weights  were 
employed.  These  were  exponential  smoothing  and  autoregressive  time-series  models. 

4.  Exponential  Smoothing  (ES):  The  exponential  smoothing  (ES)  model  is  a weighted 
moving  average  model,  but  with  a set  of  weights  that  declines  exponentially.  i he 
forecast  for  a loss  rate  one  period  ahead  using  the  ES  method  is 


X +1  - (l-a)Xt  + (l-a)aXt_1 
+ (l-a)a^X  2 + • • • 


(4) 


**Brown,  R.  G.  Smoothing,  forecasting,  and  prediction.  Englewood  Cliffs,  NJ: 
Prentice-Hall,  Inc.,  1963. 

sThe  OP-130  method  uses  all  historical  data  available  to  compute  its  loss  rate 
estimate.  For  example,  to  estimate  the  FY78  rate,  it  would  use  FY69-FY77  rates.  Under 
this  scheme,  the  FY77  rate  would  have  nine  times  the  weight  or  importance  of  the  FY69 
rate.  To  obtain  more  than  one  estimate  for  validation  purposes,  however,  the  OP- 1 30 
method  was  represented  by  the  model  using  only  7 years  of  historical  data. 
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where  a is  a fraction  between  0 and  I.  The  weighting  pattern  may  be  adjusted  by 
selecting  different  values  of  a.  When  a is  small,  the  weight  given  to  the  current 
observation  is  large  and  successive  weights  decline  rapidly.  On  the  other  hand,  if  a is 
large,  little  weight  is  given  to  the  current  observation  and  subsequent  weights  decay 
slowly. 

In  practice,  the  form  of  the  ES  model  in  equation  (4)  is  not  suitable  because 
computation  of  the  forecasts  requires  an  infinite  number  of  previous  observations.  The 
following  expression  is  used  to  compute  the  ES  estimate  for  year  t + I as  function  of  the 
loss  rate  for  year  t and  the  prior  ES  estimate  for  year  t:6 


Xt+1  = (1-a)Xt  + aXt  • (5) 


This  model  was  run  for  each  grade/YCS  cell  using  values  of  a from  0.0  to  1.0  in 
increments  of  0.1.  The  "best"  ES  model  for  each  cell  (the  "best"  value  of  a)  was  selected 
on  the  basis  of  minimum  forecast  error  (see  section  below  on  Selecting  the  "Best" 
Forecasting  Technique). 

5.  Autoregressive  Time  Series  (ARTS):  In  an  ARTS  model,  the  loss  rate  estimate  is 
still  a function  of  past  observations,  but  the  weights  given  to  past  observations  are 
computed  to  minimize  some  function  of  forecasting  error,  rather  than  being  selected  a 
priori  by  the  analyst.  The  form  of  the  estimate  is 


Xt+1  ' + ®2Xt  + 


+ 0 

n+2  t-n 


(6) 


where  the  are  the  parameter  estimates  determined  from  the  regression. 

Two  regression  approaches  were  used  to  estimate  the  parameters  in  (6).  The  first 
approach  was  the  ordinary  least  squares  (OLS)  regression.  This  method  finds  the 
parameter  estimates  that  minimize  the  sum  of  the  squared  errors  between  the  loss  rate 
estimates  and  observed  values 


<Xi- 
i=  t-n 


V‘ 


(7) 


Although  computational  requirements  for  OLS  regression  are  more  than  those  for  previous 
methods,  they  are  not  excessive.  Given  certain  assumptions  about  the  loss  rate 
distributions,  the  method  also  supplies  estimates  with  desirable  statistical  properties. 
There  are  several  reasons  to  look  beyond  OLS  in  our  case,  however.  First,  OLS  estimates 
are  sensitive  to  disturbance  by  outlier  data~any  data  points  that  differ  markedly  from  the 
rest  of  the  historical  data.  Second,  the  criterion  used  to  select  a "best"  forecasting 

* 

6 Nelson,  C.  R.  Applied  time  series  analysis  for  managerial  forecasting.  San 
Francisco:  Holden-Day,  Inc.  1973. 
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method  will  be  minimum  mean  absolute  forecast  error --it's  more  important  to  come  as 
close  as  possible  each  year  than  coming  as  close  as  possible  in  the  "worst"  year.  There  is 
another  regression  method  that  directly  minimizes  mean  absolute  forecast  error  and  that 
is  less  sensitive  to  outliers — minimum  absolute  deviation  (MAD)  regression,  a special  case 
of  constrained  regression.7  This  method  minimizes  the  sum  of  the  absolute  values  of 
errors  between  loss  rate  estimates  and  historical  rates 


I 


i=t-n 


X -X. 

i x 


(8) 


MAD  estimates  can  be  computed  using  a linear  programming  (LP)  software  package,  as 
was  done  here.  Computational  requirements  are  larger  than  for  previous  methods— a 
linear  program  must  be  solved  for  each  loss  rate  estimate— but  efficient  use  of  an  LP 
package  or  a special-purpose  program  limits  computational  requirements  to  an  acceptable 
level. 

The  OLS  and  MAD  regression  approaches  were  tried  with  an  ARTS  model  using  I,  2, 
and  3 years  of  historical  data  to  forecast  a second,  third,  and  fourth  year  respectively. 
(Use  of  more  than  3 years  would  be  statistically  unacceptable  as  it  would  reduce  the 
number  of  degrees  of  freedom,  already  small,  to  only  one.) 

6.  Time  Trend  (T);  While  few  of  the  loss  rate  series  examined  appeared  to  follow  a 
secular  or  time  trend,  the  method  of  relating  loss  rates  to  a nonstationary,  linear,  time 
trend  rather  than  past  loss  rates  was  tried.  This  involved  regressing  loss  rates  against 
relative  time  (1, 2, ...9)  as  indicated  by 


t+1 


= a + Bt 


(9) 


Selecting  the  "Best"  Forecasting  Technique 

Results  for  each  loss  rate  estimating  technique  were  calculated  for  each  grade/YCS 
cell  using  the  9 years  of  historical  data  available.  A technique  was  judged  superior  to 
another  if  it  had  a lower  mean  absolute  error  (MAE).  The  MAE  was  computed  by  summing 
the  absolute  deviations  of  a technique's  estimated  values  from  the  corresponding  actual 
loss  rates  over  time  and  dividing  by  the  number  of  years  forecast.  (The  number  of  years 
forecast  differed  across  techniques  as  some  methods  require  more  data  than  others  to 
produce  a forecast.) 

Table  3 summarizes  the  techniques  described  above  and  indicates  the  number  of 
estimates  made  for  validation  purposes  for  each  period. 


7Charnes,  A.,  & Cooper,  W.  W.  Management  models  and  industrial  applications  of 
linear  programming,  Vol,  I & II.  New  York:  John  Wiley  and  Sons,  Inc.,  1961. 
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Table  3 


Loss  Rate  Techniques  Examined  and  Number 
of  Validation  Estimates  Made  with  Each 


Technique 

Abbreviation 

Number  of  Estimates 

Naive  or  Previous  Year 

PY 

8 

Simple  Moving  Average 
(3  years) 

SM3 

6 

Simple  Moving  Average 
(4  years) 

SM4 

5 

Simple  Moving  Average 
(5  years) 

SM5 

4 

Simple  Moving  Average 
(6  years) 

SM6 

3 

Simple  Moving  Average 
(7  years) 

SM7 

2 

Weighted  Moving  Average 
(3  years) 

WM3 

6 

Weighted  Moving  Average 
(4  years) 

WM4 

5 

Weighted  Moving  Average 
(5  years) 

WM5 

4 

Weighted  Moving  Average 
(6  years) 

WM6 

3 

Weighted  Moving  Average0 
(7  years) 

WM7 

2 

Exponential  Smoothing 

ES 

8 

Autoregressive  Time  Series-OLS 
(1  year) 

ARTSI-OLS 

8 

Autoregressive  Time  Series-OLS 
(2  years) 

ARTS2-OLS 

7 

Autoregressive  Time  Series-OLS 
(3  years) 

ARTS3-OLS 

6 

Autoregressive  Time  Series-MAD 
(1  year) 

ARTS  1 -MAD 

8 

f\ 

Autoregressive  Time  Series-MAD 
(2  years) 

ARTS  1 -MAD 

1 

Autoregressive  Time  Series-MAD 
(3  years) 

ARTS3-MAD 

6 

Time  Trend 

T 

9 

aThe  weighted  moving  average  using  7 years  of  data  represents  the  OP- 1 30 
technique. 


8 


RESULTS 


The  Navy  Officer  Corps  is  divided  into  three  broad  groups  or  "communities."  The 
unrestricted  line  officers  (URL),  the  restricted  line  and  staff  corps  officers  (RL/SC),  and 
limited  duty  officers  (LDO).  As  the  largest  of  the  three  communities  with  nearly  60 
percent  of  the  active  officer  force,  the  URL  is  comprised  of  four  general  warfare 
specialities:  surface,  submarine,  aviation  (pilot  and  flight  officer),  and  special  warfare. 

Loss  rate  techniques  were  evaluated  for  the  total  URL  by  grade  and  YCS  cell.  The 
best  technique  in  each  cell — that  which  has  the  smallest  mean  absolute  error  (BEST 
MAE) — is  listed  in  Table  4.  Analysis  of  the  results  addresses  three  questions:  What  are 
the  best  forecasting  techniques?  How  accurate  are  they?  How  much  of  an  improvement 
in  forecast  accuracy  do  the  new  techniques  provide? 

Table  4 is  summarized  in  Table  5,  which  displays  the  distribution  of  BEST  MAEs  by 
grade.  It  indicates  that  the  ARTS3-MAD  technique  has  the  minimum  MAE  in  38  of  56 
(68%)  grade/YCS  cells.  Alternatively,  the  previous  technique,  WM7,  produces  the  best 
results  in  only  6 cells  (10.7%).  Finally,  of  the  13  techniques  examined,  three  (ARTS2- 
MAD,  ARTS3-MAD,  and  WM7)  accounted  for  nearly  90  percent  of  the  lowest  MAEs. 
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Table  4 


Best  Loss  Rate  Forecasting  Techniques  (Minimum  MAE) 
by  Grade/YCS  for  Total  URL 


Grade 

YCS 

Technique 

Mean  Absolute  Error 

CAPT 

20 

ARTS3-MAD 

.0042 

21 

ARTS3-MAD 

.0052 

22 

ARTS3-MAD 

.0192 

23 

ARTS3-MAD 

.0082 

24 

WM7 

.0110 

25 

ARTS3-MAD 

.0106 

26 

WM7 

.0218 

27 

ARTS3-MAD 

.0190 

28 

ARTS3-MAD 

.0032 

29 

WM3 

.0509 

CDR 

14 

WM7 

.0000 

15 

ARTS3-MAD 

.0024 

16 

ARTS3-MAD 

.0031 

17 

ARTS3-MAD 

.0022 

18 

ARTS3-MAD 

.0035 

19 

ARTS3-MAD 

.0114 

20 

ARTS3-MAD 

.0125 

21 

SM7 

.0029 

CDRF 

20 

WM7 

.0084 

21 

ART53-MAD 

.0084 

22 

ARTS3-MAD 

.0154 

23 

ARTS2-MAD 

.0189 

24 

ARTS3-MAD 

.0014 

25 

SM6 

.0143 

26 

ARTS3-MAD 

.1735 

LCDR 

8 

ARTS3-MAD 

.0103 

9 

ARTS3-MAD 

.0017 

10 

ARTS3-MAD 

.0009 

II 

ARTS2-MAD 

.0007 

12 

ARTS3-MAD 

.0032 

13 

ARTS3-MAD 

.0020 

14 

ARTS3-MAD 

.0017 

15 

SM7 

.0046 

LCDRF 

14 

ARTS3-MAD 

.0071 

15 

WM5 

.0081 

16 

ARTS3-MAD 

.0057 

17 

ARTS3-MAD 

.0169 

18 

WM6 

.0159 

19 

WM7 

.02CC 

20 

ARTS3-MAD 

.10°. 

LT 

3 

ARTS3-MAD 

.0726 

4 

ARTS3-MAD 

.0141 

5 

ARTS3-MAD 

.0111 

6 

ART52-MAD 

.0109 

7 

ARTS3-MAD 

.0049 

8 

ARTS3-MAD 

.0358 

9 

ARTS2-MAD 

.0604 

10 

ARTS3-MAD 

.1224 

II 

ARTS3-MAD 

.0580 

LTJG 

1 

ARTS2-MAD 

.1293 

2 

ARTS3-MAD 

.0180 

3 

WM7 

.0343 

4 

ARTS3-MAD 

.0322 

5 

ARTS3-MAD 

.1442 

ENS 

0 

ARTS3-MAD 

.0043 

1 

ARTS3-MAD 

.0016 
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Table  5 


Distribution  of  Best  Techniques  (Minimum  MAE)  Over  YC5  by  Grade 


GRADE 


PY  SM3  SM4  SM5  SM6  SM7  WM3  WM4  WM5  WM6  WM7  T 


ES  ARTSI-ARTS2-  ARTS3-  TOTAL 

MAD  MAD  MAD 


CAPT 

CDR 

CDRF 

LCDR 

LCDRF 

LT 

LTJG 

ENS 

TOTAL 


(0.0)  (0.0)  (0.0)  (0.0)  (1.8) 


(3.6)  (1.8)  (0.0)  ( i .8)  (1.8) 


(10.7)  (0.0)  (0.0)  (0.0) 


(10.7)  (67.9)  (100.0) 


PERCENT  OF 
TOTAL 


To  assess  the  accuracy  of  the  best  forecasting  techniques,  the  grade/YCS  cells  were 
grouped  by  size  of  the  minimum  MAEs  achieved.  The  groupings  were: 

Group  MAE  Range 

0 0.0000-0.0100 

1 0.0101-0.0200 

2 0.0201-0.0300 

3 0.0301  and  greater 

A MAE  of  0.0100  (Group  0)  suggests  that,  "on  the  average,"  the  loss  rate  estimate  will  lie 
in  a range  of  ± 0.0100  around  the  actual  loss  rate  value.  For  example,  if  the  actual  loss 
rate  were  0.0200,  the  estimate  is  likely  to  be  somewhere  between  0.2100  and  0.1900. 
Hence,  these  "ranges"  of  MAE  provide  a sense  of  relative  confidence  in  loss  rate 
forecasts. 

As  Table  6 shows,  in  nearly  half  of  the  cells  forecasted,  the  minimum  MAE  value  fell 
in  Group  0 (less  than  0.0100).  More  important,  roughly  three-quarters  of  the  cells  had  a 
MAE  of  0.0200  or  less. 

Several  of  the  least  accurate  forecasts  (MAE  greater  than  0.0300)  were  found  in  YCS 
cells  that  represent  the  "flow"  or  promotion  points  during  an  officer's  career.  These  flow 
points  change  frequently  due  to  the  effect  of  specific  promotion  plans.  These  changes  can 
result  in  erratic  cell  sizes  over  time  and,  in  turn,  highly  variable  loss  rates.  As  an 
example  of  this  problem,  see  LT  3,  LT  8-11,  and  LTJG  4-5  in  Table  4.  Other  relatively 
inaccurate  estimates  were  produced  in  junior  grade  (LT  and  below)  cells  where  the  number 
of  losses  varied  widely  relative  to  strength  as  a result  of  policy  changes  designed  to  adjust 
the  force  to  meet  future  needs.  As  an  example,  in  ENS  I,  a changing  release  from  active 
duty  (RAD)  policy  caused  loss  rates  to  vary  from  .0230  to  .1250  in  a 4-year  period  (FY72- 


To  reduce  complexity  and  data  processing  costs  in  inventory  projection  models,  it  is 
desirable  to  minimize  the  number  of  loss  rate  forecasting  techniques  used.  Ideally,  the 
same  technique  would  be  used  for  all  forecasts.  Since  ARTS3-MAD  was  the  best 
technique  in  38  of  56  cells,  an  examination  was  made  to  determine  if  ARTS3-MAD  might 
be  an  acceptable  "second  best"  estimator  in  the  other  18  cells  as  well.  This  involved 
computing  weighted  averages  of  the  BEST  and  the  ARTS3-MAD  MAEs.  To  indicate 
relative  importance  of  each  error,  the  weight  assigned  to  a cell's  MAE  was  its  FY77  begin 
strength.  This  produced  average  BEST  and  ARTS3-MAD  MAEs  of  .0135  and  .0166, 
respectively.  This  23  percent  increase  in  average  MAE  (from  .0135  to  .0166)  suggests  the 
extent  of  the  forecast  accuracy  sacrificed  by  reducing  data  processing  costs.  The  ARTS3- 
MAD  MAE  of  .0166,  however,  indicates  that  ARTS3-MAD  is  still  a rather  accurate 
forecasting  method.  OP- 1 30  has,  in  fact,  elected  to  minimize  data  processing  costs  and 
assure  increased  accuracy  by  using  ARTS3-MAD  as  its  sole  loss  rate  forecasting 
technique. 

Finally,  while  the  ARTS3-MAD  method  was  found,  in  most  cases,  to  be  a better  loss 
rate  estimator  than  the  previous  method  (WM7),  it  is  still  important  to  determine  how 
much  of  an  improvement  the  new  technique  realized.  Weighted  averages  of  the  MAEs 
produced  by  the  two  techniques  in  each  cell  gave  an  average  error  of  .0166  for  ARTS3- 
MAD  and  .0469  for  WM7.  This  implies  that  the  change  from  the  old  to  the  new  technique 
resulted  in  an  average  reduction  in  forecast  error  of  nearly  65  percent.  To  illustrate  this  % 

improvement,  the  new  technique  reduces  mean  error  for  projected  LT  losses  by  418  people 
out  of  a mean  loss  of  2000  people  each  year.  This  improvement  will  allow  more  accurate 
accession  plans,  as  well  as  a better  estimate  of  the  promotion  base  for  LCDRs. 
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Table  6 


Relative  Accuracy  of  Best  URL  Loss  Forecasts  by  Grade 
(Distribution  of  Techniques  by  MAE  Groupings) 


MAE  GROUP 
(Range) 

0(0.0000-0.0100) 

1 (0.0101-0.0200) 

2 (0.0201-0.0300) 

3 (0.0301  and 

greater) 

Total 


Number  of  Best  Forecasts  by  Grade 


CAPT  CDR  CDRF  LCDR  LCDRF  LT 


Total 


LTJG 

ENS 

N 

Percent 

0 

2 

26 

46.4 

1 

0 

16 

28.6 

0 

0 

2 

3.6 

4 

0 

12 

21.4 

5 

2 

56 

100.0 

t 


CONCLUSIONS 

1.  The  3-year  autoregressive  time  series  model  (ARTS3-MAD)  produces  substantially 
more  accurate  loss  rate  estimates  than  the  existing  OP- 1 30  method. 

2.  Although  other  techniques  may  produce  better  estimates  in  isolated  cases,  the 
ARTS3-MAD  technique  can  conveniently  be  used  in  all  grade/length  of  service  cases  while 
still  increasing  accuracy  relative  to  the  OP- 1 30  method. 


# 


tf 
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